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Abstract 


After a survey of current methods of studying lexical acquisition, 
none of which are really suitable for studying how specific words are 
acquired and used by adult learners and how such usage differs from that 
of native speakers, a method is proposed for the study of such qualitative 
aspects of second language lexical acquisition. Learners are required to 
fill in the blanks in short texts, or "discourses", which as far as possible 
are so constructed that there is only one lexical item which would be 
judged an appropriate filler by native speakers. The main advantages 
of this technique are that it can be used i) to test specific hypotheses 
about learners' interlanguage and ii) to compare different populations, 
including both learners and native speakers. Certain difficulties that 
arise in item construction and data analysis are also discussed in detail, 
particularly the problem of limiting both syntactic and situational possi- 
bilities, the significance of inappropriate responses, and the wording of 
questionnaire instructions. 
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Discourse — Completion as a Technique for Studying 
Lexical Features of Interlanguage 


E. Levenston and Sh. Blum 


Department of English and Center for Applied Linguistics, 
The Hebrew University of Jerusalem 


1. Introduction 


Two aspects of second language (L2) lexical acquisition must be dis- 
tinguished: quantitative and qualitative. ‘Quantitative study is concerned _ 
to establish how many words are acquired, in what sequence, and at what 
rate. 'Qualitative' study concentrates on specific words or lexical items, 
and investigates both how and when they are acquired and the stages by 
which the learner gains complete control of all shades of meaning. To date 
little work has been carried out of either kind; research on L2 acquisition 
has concentrated on grammar and phonology. 


One reason for the lack of research in lexical acquisition may be the 
absence of contrastive lexical studies, which are probably a necessary 
precondition: “hardly anything of significance has been attempted in the 
differential comparison of vocabulary" (Lewis, 1974). The existence of 
bilingual dictionaries inno way contradicts this verdict. Though they may 
contain much of the necessary raw data, it is scattered by the whim of 
alphabetical order, and not systematically set forth by lexical sets, or 
semantic fields. Another reason for lack of research may be the shortage 
of suitable research instruments, designed for testing specific hypotheses 
about processes of lexical acouisition. Nearly all the research to date has 
been concerned with second language — or "interlanguage'' — performance 
in general. Data about Jexical usage, no matter how elicited, is often in- 
cluded in overall assessments of the level of second language proficiency, 
and not studied as an end in, itself. 


Our own research interests are both exclusively lexical and "qualitative". 
We are interested in how L2 learners use specific lexical items, and how such 
usage differs from native speakers' usage. Such an interest in specific items 
derives from error analysis of learners! "textual data". Insights into the 
reasons underlying learners' usage suggest specific hypotheses about pro- 
cesses of lexical acquisition. These hypotheses must then be tested by 
further study of learners' usage under controlled conditions. 


The purpose of this article is to suggest an appropriate research 
technique which will provide such conditions for hypothesis testing; examples 


a 


of the kinds of hypotheses we have in mind are described below in section 
2.1. However, before analysing in detail the merits and deficiencies of 

the research techniques we have been using, it seems appropriate to survey 
other research concerned with eliciting L2 performance in general and to 
consider how far the techniques used might be suitable to our own aims in 
studying lexical performance. 


The data collection techniques used in previous studies of inter- 
language, whether used alone or in combination, can be divided into four 
main groups: 


i) "pictorial clues", i.e. pictures used to elicit either oral responses 
(Dulay and Burt, 1974; Cohen, 1974), or written ones (Barik and 
Swain, 1975), or both (Bailey, Madden and Krashen, 1974). 


ii) translation — from L1 to L2 as a test of grammatical competence 
(Taylor, 1975) or from L2 to L1 as a comprehension task and 
from Li to L2 as a production task (Swain, Dumas and Naiman, 
1974). Translation has also been used in combination with pictorial 
clues (Varadi, 1973) and as a device to ensure comprehension on 
a sentence completion task (Ickenroth, 1975). 


iii) imitation — since psycholinguists claim that correct imitation of 
model sentences in the mother tongue involves both lexical and 
syntactic processing (Ervin-Tripp, 1970), it has also been used 
recently to study second language performance (Naiman, 1974; 
Markman, Spilka and Tucker, 1975). 

{ 

iv) spontaneous speech — taped interviews have been used both for 
asséssing second language proficiency at a given point in time 
(Selinker, Swain and Dumas, 1975) and as one of a series of 
data-collecting techniques in longitudinal studies (Cazden et al. 
1975). A variation a thie technique is the structured interview 
devised by Selinker (1969)>. 


Pictorial cues are of course more appropriate for research with 
children than with adults; our own research population were largely uni- 
versity students. Both pictures and spontaneous speech also have the 
disadvantage, from our point of view, that they do not easily permit com- 
parison of learners' with native speakers' usage. Individual differences 
of personality and imagination will produce a wide range of responses even 
to the same picture. Both pictorial cueg and spontaneous speech may be 
useful for producing preliminary textual data, on which hypotheses about 


lexical acquisition can be based. But for testing hypotheses about lexical 
acauisition and communication strategies, we need precise comparison of 
usage in identical or near-identical contexts. Such comparisons may occur 
in response to pictorial cues or in taped interviews, but they are not subject 
to experimental control. 


For hypothesis testing the great drawback of spontaneous speech is 
that you cannot avoid avoidance. If the learner is free to express himself 
Spontaneously in any way, and on any subject he chooses, he is also free 
to avoid all those aspects of lexical usage which most interest the experi- 
menter. Only the most highly structured interview can provoke responses 
specific enough for the kind of research we are interested in. 


Translation from the second language into the learner's mother tongue 
can supply data only about his level of comprehension. For information 
about his second language usage we need examples of his ability to translate 
from his mother tongue into the second language. The trouble with trans- 
lation as a research techniaue is that it invites "transfer", or "interference". 
This is true for native speakers as well as learners of the target language; 
by no means all native speakers are capable of producing normal usage 
when asked to translate. For the second-language learner we can certainly 
learn what regular patterns of translation-eauivalence he has set up, but 
this may tell us little about deviations from acceptable usage. 


About imitation there is little to be said. It has not hitherto been used 
for research specifically directed to studying lexical acauisition. And it is 
not clear precisely how relevant it is for comparing the behaviour of learn- 
ers and native speakers. 


There is a fifth approach to L2 lexical acquisition which deserves con- 
sideration: the use of what we prefer to call "discourse-completion", one 
variety of those controlled elicitation procedures which have been described 
as "experimental methods whereby grammatical and semantic inquiry can 
be put on a satisfying objective basis" (Greenbaum and Quirk, 1970; 1). In 
two recent articles we have discussed specific aspects of the acquisition 
of Hebrew vo¢abulary (Levenston and Blum, 1°77; Blum and Levenston, 1977) 
where the relevant data were obtained through techniques of elicitation. The 
purpose of the current article is to analyse in detail the advantages and dis- 
advantages of these techniques. Our hope is that a better understanding of 
the practical difficulties of studying vocabulary acquisition — by which we 
mean both the words learnt and the uses and meanings attached to them at 
any given stage of the learning process — will encourage more scholars to 
try their hand at investigating this vital but neglected area of L2 acquisition. 


2. Discourse-Completion 


Essentially we used discourse completion exercises, what Quirk 
and Greenbaum call "composition tests". The subjects, both learners 
and native speakers, were presented with a series of minimal discourses — 
usually a single sentence, but occasionally two sentences. The term 
"discourse" is used here to refer to a coherent stretch of speech or writ- 
ing which contains enough information to enable the reader to supply the 
situational context in which it took place. ''Sentence- completion", a 
more familiar term from language teaching and testing, would not be ap- 
propriate; we are not concerned with the grammatical or orthographic 
properties of the sentence, and not limited to one sentence per item. 
And neither "text" nor "utterance'' — possible alternatives to "discourse" — 
suggest the degree of coherence and potential contextualization which are 
a necessary feature of each item. In each discourse, then, there was a 
blank to be completed. The instructions were to fill the blank with one 
word, the first that came to mind. And the discourses were designed to 
restrict as far as possible the number of acceptable alternatives; in the 
ideal discourse there would be only one word which a native speaker would 
be likely to supply. 


2.1. Advantages 
This brings us to the first two advantages of the method: 


Advantage 1 — Testing specific hypotheses 


In our two previous articles we have discussed the ways in which a 
speaker's first language semantic competence is relevant to his L2 lexical 
acquisition. In particular we have examined the relevance of his control of 
hypo nymy, synonymy, converseness, and register distinctions. The various 
hypotheses were tested with reference to those specific words for which the 
sentence completion exercises were devised. 


It was postulated, for instance, that where there were two words for a 
particular concept, one register-restricted and the other unmarked for 


register, the learner would first acquire the unmarked term, usually taught ~~~ 


first, and use it in all contexts, even though he may demonstrate passive 
recognition knowledge of the marked term. This was tested for Hebrew 
rahac ("wash" — unmarked) and natal ("'wash'"' — religious register, i.e., 
as a ritual act), with a sentence whose English translation was as follows: 
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According to Jewish tradition, one must one's 
hands before a meal. 


The data supported the hypotheses — 85% of the native speakers used 
the marked form, whereas among the learners who knew both words only 
38.2% used the marked natal and 60% preferred the unmarked rahac. 


Advantage 2 — Comparing different populations 


The most important comparison is between the usage of native speakers 
of a language and that of learners. And the need to establish native usage is 
not a luxury; the precision and status labélling of dictionary entries may 
obscure the vagueness and indeterminacy with which language is actually | 
used. We must therefore check our hunches about the "only word acceptable" 
to native speakers by means of empirical experiment. Although for example, 
the majority of native speakers of Hebrew did use natal in the above sentence, 
and comparison with learners' usage was therefore demonstrably valid, we 
were surprised to find as high a proportion as 12.5% of native speakers using 
rahac. 


Other populations can also be compared by the same questionnaire. 
In fact, it may be poss ble — by extensive testing of different kinds of native 
speakers — to find out what the factors are that lead to the kind of register 
insensitivity displayed by the aforementioned 12.5%. We have recently tried 
administering our original questionnaire to culturally disadvantaged Hebrew- 
speaking children; preliminary results indicate that they are not insensitive 
to register distinctions. Similarly, one can compare the results of learners 
at different stages — either the same learners at different points in time, or 
different learners at the same point in time — and gain some understanding 
of the chronological development of lexical acquisition. 


It should also be noted that the initial testing of all discourses on native 
speakers is mandatory. Failure by native speakers to agree on a single most 
acceptable word indicates the need to refine (or reject) the discourse before 
using it to test learners® behaviour. 


Advantage 3 — Ease of analysis and administration 


A written questionnaire, consisting of 20 minimal discourses, can be 3 
speedily administered to large numbers of subjects — especially university 
students during or between lectures — in a matter of minutes. No special 
skills are required of the research assistant charged with supervising the 
administration. And results can be speedily tabulated. For each discourse 
there may be a variety of answers, including those forecast by the hypotheses. 

~ \ 
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The number of occurrences of each answer provides immediate evidence of 

the direction in which the research is heading. ‘the frequency with which expectea an- 
gwers occur will confirm or disconfirm the hypotheses. And each unexpected 

answer, however infrequent, provided by a learner provides further food 

for thought about second language communication strategies that use lexis. 


Advantage 4 — Use with other tests 


In our own research we have been using the discourse-completion 
"composition" tests, in coordination with a simple translation test. Those 
subjects who are being studied for their use and knowledge of a second 
language are asked to translate into their mother tongue a list of items, 
including all those which prior research has shown most likely to be used 
in the composition tests by native speakers. Comparison can then be made 
between the learners passive/recognition knowledge of vocabulary — as re- 
vealed by his ability to recognize and translate items out of context — and 
his active/production knowledge — as revealed by his ability to recall the 
same items, when known passively, for use in appropriate contexts. 


It would also be interesting to compare the 1 >sults of the written 
composition tests with data obtained from an oral version of the same 
discourse-completion exercise. We have assumed hitherto that the test is 
a ''projective’ measure, i.e., that what subjects do write is a reliable indi- 
cator of what they would say. But oral production can be tested directly. 
We could let subjects hear the incomplete sentence and ask them to utter 
the missing word. It might prove necessary to redesign each discourse in 
such a way that the missing item always occurred in final position, though 
this might limit the items which could be tested. In all other respects the 
initial hypotheses and experimental procedures would remain unchanged. 
The translation test could also be administered orally. The only reeson for 
preferring a written version of the test is that it is much less bother. 


Advantage 5 “ts Degree of contextual control 


The most obvious yet perhaps the greatest advantage of discourse- 
completion is the degree of contextual control it gives the experimenter. 
"Contextual" is used here to refer to the linguistic context, or, more pre- 
cisely, to the situation implied by the lexical material contained in the 
discourse. The greater the extent to which this implied situation is restricted 
and defined, the greater the constraints upon linguistic choice available when 
any given lexical item is deleted. (See example 3 below, and the suggested 
emendation. ) In our own research we have striven for maximum contextual 


restriction, in order to test usage of specific lexical items. It is also pos- 
sible, by relaxing the degree of contextual control in a particular discourse, 
to test usage in areas of vocabulary. In either case, prior testing with 
native speakers will indicate the level of contextual constraint achieved and 
the range of acceptable responses. 


3. Difficulties 


In designing discourse-completion tests and interpreting the results 
we have encountered a number of difficulties, some expected but some totally 
unexpected. It seems useful, therefore, to list some of the lessons we have 
learned about research procedure. We have not solved all the problems, but 
we can warn intending experimenters of the difficulties they are bound to 
run into. 


a. Discourse design 


Firstly, each discourse must be carefully designed so that only one 
grammatical structure or "part of speech" is syntactically admissible in 
the context. Otherwise, hypotheses concerning specific words that are not 
used as more than one part of speech cannot be tested. And most vocabulary 
items are probably restricted to single parts of speech. The following dis- 
course is an example of an unsatisfactory item; as the suggested completions 
show, it may be completed by either an adjective or the continuous tense of 
a verb. 


1) He wants to leave Israel. He can't find a good job, and he has no 
friends. In short, I don't think he's here. 


hap py/ staying 


Secondly, it must be unambiguously clear what time and tense relation- 
ships affect the choice of the test item. The following example illustrates 
the range of possibilities that may unfortunately become available if time and 
tense are not controlled. 


2) They gave him a good job. He a lot 


likes /needed/will earn 


of money. 


i 
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Thirdly, the context must be specified in sufficient detail to limit the 
referential and conceptual possibilities of the test-item within the desired 
limits: 


3) When he heard that he was the father of a son, he was very 
happy/ upset. 


Such a discourse might well be improved by inserting the phrase "after 
six daughters". 


Finally, as pointed out in advantage 5, the smaller the number of ac- 
ceptable lexical items the discourse is designed to elicit, the more important 
these parameters — grammatical, situational and referential — become. If, 
for example, you are interested in semantic fields rather than specific items, 
then it does not matter if there is not such strict control of the grammatical 
environment. 


b. Inappropriate responses 


No matter how precise the instructions given when administering the 
test, there will always be some responses which make very little sense be- 
cause subjects ignore some aspects ofthe instructions. In our experience 
this is true, in varying degrees, of both native speakers and second-language 
learners. Both are capable of misreading, no ener how long or short the 
discourse. 


4) otan > itan 


“a, 


oe (them > with them) 


Misreading ‘otan' for 'itan' simply because the letters look 
alike in Hebrew, gave subjects a whole new range of lexical 
possibilities we of course never thought of. 


Both are liable to ignore grammatical constraints, no matter how carefully 
the discourse is designed. ' 


5) He was ready to accept the responsibility that had been 
upon him. 


given/ placed 


Second language learners — though not native speakers — do not always con- 
sider the implications of the total discourse, particularly when it consists of 
more than one sentence. The word they supply may be acceptable in the con- 
text of the sentence in which it occurs, but make little sense if the total dis- 
course is taken into consideration. We are not sure why this occurs. It may 
result from the general artificiality of the combined sentences, or it may 


10 ate 
pet 


ladies] 
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stem from a tendency by second language learners to simplify the test situa- 
tions in which they find themselves. 


6) They gave him a good job. He a lot of money. 
earned/spent/ found 


The same phenomenon may also occur with discourses consisting of a single 
sentence, where the learner simply ignores whole sections of the sentence. 


7) They expelled him from the university because he 
in all the exams. 
failed/ succeeded 
None of these inappropriate responses marg the general usefulness of 
discourse-completion in lexical acauisition peau most subjects do respond 
more or less along the lines predicted. It is cléarly impossible, however, to 
achieve 100% relevant responses. When a discourse was designed to elicit 
a specific word from native speakers, we reckoned agreement by 75% as satis- 
factory. With anything less than 75% we blamed the discourse, not the sub- | 
jects. But the figure was purely arbitrary; more research is needed to 
establish accurately the correct cut-off point between appropriate and in- 
appropriate responses. 


c. Other appropriate responses a 


that up to 25% of our subjects regularly provided inappropriate responses. 
Those responses, especially from native speakers, which failed to confirm 
our hypotheses were normally appropriate to the discourse. Some degree of 
acceptable synonymy is difficult to avoid when you are working with minimal 
discourses. And as pointed out above (see Advantage 2) some subjects do not 
show sensitivity to register distinctions in their first language, if the findings 
of our tests are to be trusted. 


d. Questionnaire instructions 


As with any research based on the completion of written questionnaires, 
the precise phrasing of the instructions is probably of\vital importance. Do 
you ask the subject to write down what he "would say", what he "would write", 
or just the first word that "comes to mind"? How far can our results, elicited | 
by the last of these formulae, be projected to the situations embodied in the / 


The last paragraph should not be misunderstood. We are not suggesting | 
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first two? This is itself a subject for research, using different instructions 
for the same test items with matched groups of subjects. 


Incidentally, no matter how precisely you word the instructions or 
orally brief the subjects, there will always be some who fail to do as re- 
quested. Many of our second-language learners — though ‘fewer native 
speakers — ignored the restriction to single words, and filled the gaps with 
more than one word. If, however, we had been interested in lexical areas — 
rather than lexical items, even these responses would have been relevant. 


3. In Conclusion 


«This article has stressed the value of discourse-completion designed 
to elicit unique single-word responses. These are most useful for confirm- 
however, to realize that every appropriate response, predicted or not, has 
something to tell us about learning strategies. Among the data there is very 
little dross. 
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